Multi axis motion and position controller for portable electronic displays

ABSTRACT

A virtual computer monitor is described which enables instantaneous and intuitive visual access to large amounts of visual data by providing the user with a large display projected virtually in front of the user. The user wears a head-mounted display containing a head-tracker, which together allow the user to position an instantaneous viewport provided by the head-mounted display at any position within the large virtual display by turning to look in the desired direction. The instantaneous viewport further includes a mouse pointer, which may be positioned by turning the user&#39;s head, and which may be further positioned using a mouse or analogous control device. A particular advantage of the virtual computer monitor is intuitive access to enlarged computer output for visually-impaired individuals.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This is a continuation of U.S. patent application Ser. No.09/895,765 filed Jun. 28, 2001 and U.S. patent application Ser. No.09/895,576 filed Jun. 28, 2001, both of which are currently pending,both of which are continuations of U.S. application Ser. No. 09/373,286filed Aug. 12, 1999, now U.S. Pat. No. 6,359,603, issued Mar. 19, 2002,which is a continuation of U.S. application Ser. No. 09/235,096 filedJan. 12, 1999, now U.S. Pat. No. 6,127,990 issued 10/3/00, which is acontinuation of U.S. application Ser. No. 08/563,525 filed Nov. 18,1995, abandoned.

FIELD OF THE INVENTION

[0002] The invention relates to human/computer interfaces to visual dataand more particularly to systems that must display a larger amount ofvisual data than may be conveniently displayed in a single conventionalcomputer monitor. The present invention uses virtual reality techniquesto provide instantaneous and intuitive access to large fields of visualdata, and to provide visually-impaired users with enhanced access toenlarged visual data.

DESCRIPTION OF PRIOR ART

[0003] Among the visually-impaired population, the most common approachto computer access is specialized software and/or hardware that enlargesthe image displayed on the computer monitor. This is because simplersolutions such as moving closer to the monitor, using a larger monitor,adding an optical screen magnifier, or using a spectacle-mountedtelescopic system provide either limited magnification or a very limitedviewing field. Examples of commercially-available screen enlargersinclude LP-DOS by Optelec (Westford, Mass.), Zoomtext by Ai Squared(Manchester Center, Vt.), MAGic by Microsystems Software (Framingham,Mass.), and Magnum by Arctic Technologies (Troy, Mich.). In addition,simplistic screen enlargement modules are included in both the MicrosoftWindows and Apple Macintosh operating systems.

[0004] These conventional computer display magnification solutionsoperate by magnifying the original image of a software application'soutput to a “virtual page” whose size is much larger than the physicalmonitor. For example, with a magnification of 10, a standard 8.5″×11″page would be approximately 7 feet wide by 9 feet tall. Thevisually-impaired user then operates the computer by using a mouse,joystick, or cursor keys to control which portion of the virtual page isshown on the monitor at any given point in time. Since the monitor isfixed, the user is in essence moving the virtual page across themonitor, in a manner analogous to that used in closed-circuit television(CCTV) systems for magnifying book pages.

[0005] In most cases, conventional screen magnification is performedcompletely in software running on the host computer's central processingunit (CPU). While this provides a very low-cost solution, the data to beshown on the display must be rendered in its entirety whenever the userpans to a new location within the enlarged image. This can result inlags between commanding the computer to pan and seeing the new image. Toovercome this problem, the entire virtual image can be rendered andstored in a video display buffer. Then, as the user selects a portion ofthe image for viewing, the required portion of the data can be quicklyread out of the display buffer and sent to the display device. Anexample of such a hardware-accelerated screen magnifier is the Vista byTelesensory, Inc. (Mountain View, Calif.). This technique is a form ofhardware acceleration known as image deflection.

[0006] Unfortunately, there are two basic shortcomings to theconventional approach, even with hardware acceleration. The firstproblem is spatial orientation, in that it is difficult to determinewhere on the page one's view is directed at any given time. This occursbecause the monitor does not move, and there are no other visual cues toindicate where on the virtual page one's line of sight is facing. Thisspatial orientation problem is exacerbated for high magnifications andfor portable systems employing small display monitors. For example, onestudy (Goodrich, et. al.) found mean magnifications of 15.48× for nearly100 experienced users of closed-circuit television devices. At 15×, a15″ monitor can only display about 1% of a standard 8.5″×11″ page,making most computer work essentially impossible for such users. Theproblem is further exacerbated by the emergence of graphically-intensivecomputing regimes such as Microsoft Windows and the Internet World WideWeb, where individual graphic elements may be magnified to become largerthan an instantaneous viewing window, or may be automatically generatedoutside of the user's instantaneous viewing window without the user'sawareness.

[0007] The second fundamental problem in the conventional approach isdynamic control, in that all of the various control schemes fornavigating about the page are cumbersome, confusing, and slow. This isbecause the navigation methods are unintuitive, relying on such logic as“use joystick to move cursor around screen, and when cursor reaches theedge of the screen, the next portion of document in that direction willbe displayed.” Alternatively, some screen enlargers maintain the cursorat the center of the screen, and require the user to position a desiredinsertion point over the cursor by moving the entire virtual page with amouse or joystick. In all cases, dynamic control is not onlyunintuitive, but requires use of at least one hand, which negativelyimpacts productivity, and may make use by physically-impaired usersdifficult or impossible.

[0008] Together, these spatial orientation and dynamic control problemswere termed the “field navigation” problem in the National Advisory EyeCouncil's 1994-98 National Plan (Legge, et. al.), in which the LowVision and its Rehabilitation Panel identified this area as aparticularly promising opportunity for new technologies.

[0009] One promising new technology that is now maturing is virtualreality, which is typically defined as a computer-generatedthree-dimensional environment providing the ability to navigate aboutthe environment, turn one's head to look around the environment, andinteract with simulated objects in the environment using a controlperipheral.

[0010] In a virtual reality system, the user is “immersed” in asynthetic environment, in which virtual objects can be located anywherein the user's physical space. The user views these objects by wearing ahead-mounted display (HMD), which uses an optical system to cause a tinydisplay source such as a cathode ray tube or liquid crystal display toappear as a large display screen several feet in front of the user.Since the display source (or sources in the case of two eyes) is fixedto the user's head, the display is viewable regardless of where the userpoints his line-of-sight. The user also wears a head-tracker, whichsenses the direction the user is facing, and sends this information tothe host computer. The computer uses this data to generate graphicscorresponding to the user's line of sight in the virtual environment.This approach to human/computer interfaces was first conceived by IvanSutherland in 1966 for use in military simulators, and was firstcommercialized in the form of the Eyephone head-mounted display by VPLResearch in the late 1980s.

[0011] Prior art in this area includes a wide range of relevant patentsdescribing low-vision aids, improved virtual reality systems andcomponents such as HMDs and head-trackers, but none which embody oranticipate the present invention.

[0012] In the field of low-vision aids, U.S. Pat. No. 4,227,209 issuedOct. 10, 1980 discloses an electronic sensory aid for visually-impairedusers including an image sensor and a display array, wherein the degreeof magnification provided in the display array may be adjusted bychanging the number of display elements corresponding to each sensorarray element. For use in electronic sensory aid applications requiringa large depth of focus, an improved image capture approach is disclosedin U.S. Pat. No. 5,325,123 issued Jun. 28, 1994, in which the imagingcamera includes an opaque stop with a small aperture, thus allowing themagnification to be adjusted by moving the camera towards or away fromthe object to be magnified. A non-electronic sensory aid is disclosed inU.S. Pat. No. 4,548,485 issued Oct. 22, 1985, in which an XY stage isused to move textual material across an optical viewing system thatcaptures a portion of the textual material for enlargement.

[0013] In U.S. Pat. No. 5,125,046 issued Jun. 23, 1992, and U.S. Pat.No. 5,267,331 issued Nov. 30, 1993, an improved imaging enhancer forvisually-impaired users is disclosed in which an image is captured,digitized, and electronically enhanced to increase contrast beforedisplaying the imagery. An improvement to this approach using ahead-mounted display is disclosed in U.S. Pat. No. 5,359,675, issuedOct. 25, 1994.

[0014] In the field of virtual reality systems, U.S. Pat. No. 5,367,614issued Nov. 22, 1994 to Bisey discloses a three-dimensional computerimage display system using an ultrasonic transceiver head-trackingsystem to control a three-dimensional display to cause the image tochange its perspective in response to head movements. In addition, U.S.Pat. No. 5,442,734 issued Aug. 15, 1995 to Murakami discloses a virtualreality system incorporating a head-mounted display, head-tracker, andimage processing system in which predictive tracking algorithms are usedto differentially update portions of the display field to provide morerapid updating of those portions of the display field corresponding tothe center of the user's visual field. In U.S. pat. application Ser. No.07/621,446 (pending) filed by VPL Research, Inc., a virtual realitysystem is disclose d in which spatialized audio cues are generated toprovide real-time feedback to users upon successful completion of manualtasks such as grasping virtual objects using a sensor-laden glove inputdevice.

[0015] In the head-mounted display field, U.S. Pat. No. 5,003,300 issuedMar. 26, 1991 to Wells discloses a raster-based head-mounted displaythat may be used to display an image to either eye. U.S. Pat. No.5,151,722 issued Sep. 29, 1992 to Massof discloses a video-basedhead-mounted display featuring a unique folding optic configuration sothat the device may be worn like a pair of glasses. U.S. Pat. No.5,281,957 issued Jan. 25, 1994 to Schoolman discloses a portablecomputer system incorporating a head-mounted display that may behinge-mounted to an eyeglass frame so that the display may be folded upout of the way for viewing the physical environment. A wide variety ofadditional patents in the area of specific design improvements forhead-mounted display devices exists, however, the specific head-mounteddisplay design approach employed to effect the present invention is notcritical, so long as image quality, brightness, contrast, and comfortare maintained at high levels.

[0016] In recent years, there have been several attempts made to applyhead-mounted displays to the problems of enhancing imagery forvisually-impaired users. One such effort has resulted in the Low-VisionEnhancement System (LVES) developed by Johns Hopkins University andmarketed commercially by Visionics (Minneapolis, Minn.). The LVES deviceincorporates a head-mounted display with integrated cameras and an imageprocessing system. The cameras generate an image of whatever ispositioned directly in front of the user, and the image processingsystem enlarges the image and performs enhancement functions such ascontrast enhancement. While the LVES device can provide magnifiedimagery of real-world objects to some visually-impaired users, itsuffers from several shortcomings compared to the present invention.First, the LVES does not incorporate a head-tracker to provide ahands-free means for navigating within computer data. Further, the LVESsuffers from a jitter problem exactly analogous to that experienced byusers of binoculars or telescopes. In simple terms, any jitter in theuser's line-of-sight is magnified by the same factor as the imagery,which causes the image provided to the user to appear unsteady.

[0017] U.S. Pat. No. 5,109,282 issued Apr. 28, 1992 to Peli discloses anovel image processing method for converting continuous grey tone imagesinto high resolution halftone images, and describes an embodiment of themethod applicable to presentation of enlarged imagery tovisually-impaired users via a head-mounted display. In this device, theimagery is generated by a conventional camera manually scanned acrossprinted text as is common in closed-circuit television systems for thevisually-impaired. The head-mounted display is a Private Eye byReflection Technologies (Waltham, Mass.), which employs a linear arrayof light-emitting diodes converted to the impression of a rectangulararray by means of a scanning mirror. In the disclosed device, benefitsof using a head-mounted display for low-vision access to printedmaterial in portable situations are discussed, including the largervisual field, higher visual contrast, lighter weight, and smallerphysical size provided by an HMD compared to a portable conventionaltelevision monitor. However, no connection to a computer for viewingcomputer-generated imagery is disclosed or anticipated, and noincorporation of a head-tracking device is disclosed or anticipated.

[0018] In the tracker art, a variety of tracking approaches andapplications have been conceived and constructed. U.S. Pat. No.5,373,857 issued Dec. 12, 1994 to Travers, discloses a head-trackingapproach for the yaw degree of freedom in virtual reality applicationsconsisting of a magnetic sensor disposed on a headset to produce adisplacement signal relative to angular displacement of the head setwith respect to the earth's magnetic field. A more sophisticatedapproach has been developed by the Massachusetts Institute of Technology(MIT), in which an analogous magnetic sensor is used to correct drift ina much faster differential sensor such as an accelerometer, whichsensors together provide extremely rapid response and high accuracywithin a single package. The MIT approach, believed to bepatent-pending, additionally incorporates differential sensors to detectchanges in the pitch and roll degrees of freedom, which sensors may alsobe corrected using slower absolute sensors such as liquid-filledcapacitive tilt sensors.

[0019] Also within the tracker art, a number of devices have beendisclosed which sense head movement for purposes of controllingpositioning of a cursor or mouse pointer within the viewable portion ofa conventional display monitor. U.S. Pat. No. 4,209,255 issued Jun. 24,1980 to Heynau discloses a system for pilots employing a light-emittingdiode mounted on the pilot's head, with photodiodes located the displayto sense the tapered energy field from the light-emitting diode forpurposes of determining the pilot's aimpoint within the display.

[0020] U.S. Pat. No. 4,565,999 issued Jan. 21, 1986 to King discloses acursor control system for use with a data terminal wherein a radiationsource and a radiation sensor are used to determine changes in a user'shead position for purposes of controlling cursor position on the screen.

[0021] U.S. Pat. No. 4,567,479 issued Jan. 28, 1986 to Boyd discloses adirectional controller for video or computer input byphysically-impaired users consisting of a series of mercury switchesdisposed in proximity to a user's head, wherein movements of the user'shead are sensed and converted into cursor control commands. This devicealso employs a pressure switch activated by the user's mouth which canprovide a further control signal such as that generated by clicking amouse button.

[0022] U.S. Pat. No. 4,682,159 issued Jul. 27, 1987 to Davison disclosesan apparatus and method for controlling a cursor on a computer displaythat consists of a headset worn by the user, and a stationary ultrasonictransmitter for emitting sound waves which are picked up by receivers inthe headset. These sound waves are compared for phase changes, which areconverted into positional change data for controlling the cursor.

[0023] U.S. Pat. No. 5,367,315 issued Nov. 22, 1994 to Pan discloses aninfrared-light based system that indicates head and-eye position in realtime, so as to enable a computer user to control cursor movement an adisplay by moving his or her eyes or head. The device is intended toemulate a standard mouse, thereby allowing use of the presentlyavailable software and hardware.

[0024] While the above examples demonstrate a well-developed art forcontrolling computer cursors via head movement, none disclose oranticipate application of head-controlled cursor movement within ahead-mounted display, and none anticipate an approach such as thepresent invention wherein the cursor remains fixed at a particularposition within the display while the displayed data is moved instead ofthe cursor. Movement of displayed data within a head-mounted display inresponse to head movement has heretofore been used only within virtualreality systems designed for simulating sensory immersion withinthree-dimensional computer simulations. In such applications, cursors ormouse pointers are not controlled by head movement, but are generatedwhen required through the use of a separate hand-controlled inputdevice.

[0025] While virtual reality is still a developmental technologyinvolving exotic graphics hardware, specialized software, and longintegration cycles, the concept of closing a control loop betweenhead-tracker data and HMD imagery can be implemented analogously forviewing arbitrary computer data instead of specially-constructed virtualenvironments. For normally sighted individuals, this could be beneficialby providing a large virtual computer desktop surrounding the user,which can provide simultaneous access to a larger amount of visual datathan is possible using the small virtual desktops currently provided oncommon computing platforms such as Macintosh and windows. Forvisually-impaired individuals, head-tracked HMD display techniques canbe used to conveniently access a magnified virtual page, and thus enableproductive computer use by nearly 1,000,000 new users.

SUMMARY OF THE INVENTION

[0026] It is therefore an object of the current invention to solve thefield navigation problem by combining virtual reality display techniquesoriginally developed for military flight simulators with screenmagnification techniques, in order to create a novel and intuitivedisplay interface for visually impaired users.

[0027] It is another object of the current invention to provide anintuitive computer display interface allowing the user to automaticallyachieve proper spatial orientation by directly coupling the user's headorientation to the displayed portion of a magnified virtual page.

[0028] It is a further object of the current invention to provide anintuitive computer display interface allowing the user to automaticallycontrol the position of a cursor or mouse pointer on acomputer-generated virtual page by directly coupling the user's headmovements to movements of a cursor across the virtual page, thus freeingthe user's hands for other tasks.

[0029] It is an additional object of the present invention to providehands-free instantaneous selection from between many concurrently activecomputer applications by changing one's line-of-sight from oneapplication window's virtual location to another.

[0030] It is yet another object of the present invention to provide andmaintain a cursor at a user-selectable position within the user'sfield-of-view, in order to support use of the virtual computer displayby users with arbitrary, non-central preferred retinal loci.

[0031] It is still another object of the present invention to alert theuser to events occurring outside of the user's instantaneousfield-of-view through the use of spatialized audio alerts perceived tooriginate from the direction of the event, thus causing the user to turnand look in the direction of the event.

[0032] It is yet a further object of the present invention to provideeffective operation at magnifications much greater than those possibleusing fixed monitors, by using a novel technique known as spatialcompression.

[0033] It is still another object of the present invention to provideimproved scrolling of imagery across the user's field-of-view, throughapplication of smoothing, thresholding, prediction, and driftcompensation algorithms to improve response to raw data representing theuser's instantaneous line of sight.

[0034] It is still a further object of the present invention to providea computer display for visually-impaired users that is convenient,lightweight, low-cost, minimally power hungry, and capable of portableoperation without degraded performance.

[0035] It is another object of the present invention to provide a meansfor visually-impaired users to view enlarged video imagery in real timeover an expanded field-of-regard, thus reducing jitter compared tohead-mounted closed-circuit television systems.

[0036] In accordance with the present invention, there has been deviseda “virtual computer monitor” (VCM) which is broadly comprised of ahead-mounted display means worn by the user, a head-orientation sensingmeans worn by the user, and software means for interfacing these devicesto a host computer such that the user's head orientation data isprocessed to determine which portion of an arbitrary softwareapplication's output imagery to display. By properly matching the angleof head rotation to the extent of scrolling across the magnified image,the image can be made to appear fixed in space. The particular locationof the portion of the virtual image which is actually being seen by theuser is dependent upon the direction in which the user looks. As theuser looks to the right, the portion of the virtual image being seen bythe user is to the right of the portion of the virtual image previouslybeing seen by the user. Similarly, as the user looks up, the portion ofthe virtual image being seen by the user is above the portion of thevirtual image previously seen by the user. Upon initialization of theVCM device, the user triggers calibration between the user'sstraight-ahead line of sight and the center of the virtual page. Fromthen on, the user can rotate her head left, right, up, and down tovisually scan across the page in corresponding directions. The overallimpression is analogous to a normally sighted person scanning across anewspaper page.

[0037] As applied to a computer interface device for thevisually-impaired, the VCM software provides a magnification adjustmentto allow each user to achieve adequate visual resolution withoutneedlessly reducing his instantaneous viewing field. The software alsoprovides a cursor, which nominally remains positioned at the center ofthe HMD physical field regardless of head movements so that the cursorcan be positioned anywhere upon the virtual page by turning to face thatlocation. A further adjustment allows setting the fixed cursor locationto any arbitrary position in the HMD device's physical field, so thatusers with unusable portions of their visual fields can select analternative preferred retinal loci instead of the center. A softwareselection also provides an overview display, which shows areduced-magnification image of the entire virtual page, with a boldblack box highlighting the outline of the instantaneous field within theentire field.

[0038] An additional important feature is the ability to temporarilyadjust the cursor position in real-time using a controller peripheralsuch as a joystick or mouse. This feature allows fine positioning of thecursor within the field by temporarily locking the head-tracking systemto freeze a portion of the virtual page on the physical display, whilethe controller is used to move the cursor in small increments.

[0039] An additional important feature is the ability to display imagecomponents in addition to the cursor at fixed points in the physicaldisplay, which allows menus or other icons to remain in the user'sinstantaneous viewing field at all times while scrolling across imagecontent.

[0040] An additional important feature resides in the ability to reducethe lag between a head motion and display of the new direction's imageby using image deflection, thresholding, smoothing, prediction, and anovel drift compensation technique to reduce display “swimming”, whichis caused whenever imperfect head orientation sensing causes thedisplayed image to not appear fixed in real-space.

[0041] An additional important feature resides in the ability to magnifyimages by extremely large factors using spatial field compression, wherethe displayed image is scrolled across the physical display at a fasterrate than the head is turned. This enables use by individuals withlimited head motion, and allows magnification to levels that wouldotherwise require turning completely around to view edges of the image.

[0042] An additional important feature resides in the use of a partiallyimmersive HMD, which avoids simulation sickness by allowing the user tomaintain a constant frame of reference in the physical world since realobjects can be seen around one or more edges of the display.

[0043] It is therefore an advantage of the current invention that itsolves the field navigation problem by combining virtual reality displaytechniques originally developed for military flight simulators withscreen magnification techniques, in order to provide a novel andintuitive display interface for visually impaired users.

[0044] It is another advantage of the current invention that it providesan intuitive computer display interface allowing the user toautomatically achieve proper spatial orientation by directly couplingthe user's head orientation to the displayed portion of a magnifiedvirtual page.

[0045] It is a further advantage of the current invention that itprovides an intuitive computer display interface allowing the user toautomatically control the position of a cursor or mouse pointer on acomputer-generated virtual page by directly coupling the user's headmovements to movements of a cursor across the virtual page, thus freeingthe user's hands for other tasks.

[0046] It is an additional advantage of the present invention that itprovides hands-free instantaneous selection from between manyconcurrently active computer applications by changing one'sline-of-sight from one application window's virtual location to another.

[0047] It is yet another advantage of the present invention that itprovides and maintains a cursor at a user-selectable position within theuser's field-of-view, in order to support use of the virtual computerdisplay by users with arbitrary, non-central preferred retinal loci.

[0048] It is still another advantage of the present invention that italerts the user to events occurring outside of the user's instantaneousfield-of-view through the use of spatialized audio alerts perceived tooriginate from the direction of the event, thus causing the user to turnand look in the direction of the event.

[0049] It is yet a further advantage of the present invention that itprovides effective operation at magnifications much greater than thosepossible using fixed monitors, by using a novel technique known asspatial compression.

[0050] It is still another advantage of the present invention that itprovides improved scrolling of imagery across the user's field-of-view,through application of smoothing, thresholding, prediction, and driftcompensation algorithms to improve response to raw data representing theuser's instantaneous line of sight.

[0051] It is still a further advantage of the present invention that itprovides a computer display for visually-impaired users that isconvenient, lightweight, low-cost, minimally power hungry, and capableof portable operation without degraded performance.

[0052] It is another advantage of the present invention that it providesa means for visually-impaired users to view enlarged video imagery inreal time over an expanded field-of-regard, thus reducing jittercompared to head-mounted closed-circuit television systems.

[0053] The above and other objects, features, and advantages of thepresent invention will become more readily understood and appreciatedfrom a consideration of the following detailed description of thepreferred embodiment when taken together with the accompanying drawings,which, however, should not be taken as limitative to the presentinvention but for elucidation and explanation only.

BRIEF-DESCRIPTION OF THE-DRAWINGS

[0054]FIG. 1 is a conceptual sketch illustrating operation of aconventional screen enlarger.

[0055]FIG. 2 is a block diagram of the hardware components of thevirtual computer monitor.

[0056]FIG. 3 is a conceptual sketch illustrating operation of a virtualcomputer monitor, and intuitive field navigation via head rotation.

[0057]FIG. 4 illustrates various means for configuring the virtualcomputer monitor display, including A) typical configuration, B) typicalconfiguration in combination with a blockage of the user's fovealvision, C) mouse pointer/cursor offset to a non-central preferredretinal locus, and D) Entire display field offset to be centered about anon-central preferred retinal locus.

[0058]FIG. 5 is a block diagram of the logical flow of data processingin an advanced embodiment of the virtual computer monitor.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

[0059] The system of this invention 10 concerns a computer 12 controlledby a user through conventional means such as a keyboard 14 and inputcontroller 16, and whose output is viewed on a display monitor 18.Referring to FIG. 1, the invention 10 is specifically intended for usein applications where the total amount of data to be viewed can beconfigured as a virtual display 20, which is significantly greater inextent than the amount of data that can be conveniently viewed within aninstantaneous viewport 22 provided by the display monitor 18. An exampleof such an application is when the virtual display 20 consists of alarge computer desktop running several application windows, while theamount of data that can be visually resolved within the instantaneousviewport 22 consists of a single application window. Another example ofsuch an application is when the virtual display 20 consists of aword-processing document magnified for a visually-impaired user by ascreen enlarger 24, which may consist of software or a combination ofsoftware and hardware. In either case, conventional control means suchas a keyboard 14 or input controller 16 may be used to select whichportion of the virtual display 20 is shown within the display monitor 18at any given moment, as described in the prior art.

[0060] Referring to FIG. 2, the most basic embodiment of the invention10 is achieved by implementing the display monitor as a head-mounteddisplay 26, wherein tiny display sources such as LCDs are held withinclose proximity to the user's-eyes, and optically coupled to the eyeswith a lensing system such that the image of the computer displayappears to float in space several feet in front of the user. Suchdevices are well known in the art, and are commercially available fromsources including General Reality Company (San Jose, Calif.), Optics 1(Westlake, Calif.), and Virtual I/O (Seattle, Wash.). In addition to thehead-mounted display, the user wears a head-tracker 28, which senseschanges in orientation of the user's head and reports them to thecomputer 12 to result in the perception of scrolling the instantaneousviewport 22 across the virtual display 20. Head-trackers are alsowell-known in the art, and a variety of different devices are availablefrom sources including General Reality Company, Precision Navigation(Mountain View, Calif.), and Polhemus Inc. (Colchester, Vt.).

[0061] With respect to the present invention, a wide variety ofdifferent head-mounted displays 26 and head-trackers 28 may be usedwithout affecting the fundamental operating concepts embodied therein,and many suitable devices are commercially available. For thehead-mounted display 26, it is important to provide adequatefield-of-view to ensure that a significant portion of the user's visualfield is addressed, and to provide a sufficient number of pictureelements, or pixels, so that small text can be resolved. Useful minimumsare twenty degree field-of-view and 100,000 pixels per eye, althoughthese figures are subjective. In addition, visual contrast must be high(100 to 1 or greater) for visually-impaired users. For somevisually-impaired users, maximizing contrast can become sufficientlycritical that a color display can not be used, and a black and whiteunit must be used instead. In general, parameters such as field-of-view,pixel count, contrast size/weight, cost, and other factors such asapparent image distance well-known in the art of head-mounted displaydesign may be traded-off to provide a best compromise over a variedpopulation of users, or may be traded-off to optimize performance for asingle user.

[0062] The simplest embodiment of the invention 10 uses a CyberTrack™model head-tracker 28 from General Reality Company. This model providesan output signal emulating that of a mouse, which can be read directlyby a standard Microsoft mouse driver 30 for purposes of controlling themanner in which the instantaneous viewport 22 is selected from withinthe virtual display 20. In alternative embodiments using a differenthead-tracker 28 which can not so emulate a mouse, an additional softwaremodule can be used to interpret the output of the head-tracker 28 andconvert the output into “mickeys” that emulate mouse output, or anadditional software module can adapt the output of the head-tracker 28for directly controlling scrolling of the instantaneous viewport 22without use of an intervening mouse driver. All told, a wide variety ofalternative means for converting head-tracker output into scrolling ofthe instantaneous viewport have been conceived, so the approach selectedfor the present embodiments should not be considered limitative of theinvention.

[0063] Referring to-FIG. 3, the result of the invention 10 is to providethe user with the perception that the virtual display 20 is fixed inspace in front of the user, and that the user can position theinstantaneous viewport 22 provided by the head-mounted display 26 at anypoint within the virtual display 20 merely by rotating his or her headto look in the desired direction. Because the user's nervous systemprovides proprioceptive feedback which constantly provides the user witha sense of direction, and because turning to look in a particulardirection is a natural and intuitive means for viewing objects in thatdirection, the invention 10 provides a solution for both the spatialorientation and dynamic control aspects of the field navigation problem.

[0064] Referring to FIG. 4, further detail of-the instantaneous viewportis shown. Specifically, a mouse pointer 32 is shown. The mouse pointer32 is typically maintained in the center of the instantaneous viewport22 as the viewport is scrolled, and is used to allow selection ofparticular data items by the user. Such selection may be performed byclicking a button on the input controller 16. In the present invention10, the mouse pointer may also be adjusted to remain at a non-centralposition 34 within the instantaneous viewport 22 while the viewport isscrolled. Such a non-central position 34 may be used in a case where theuser suffers from a visual impairment such as macular degeneration,which can cause the foveal (central) portion of the user's visual fieldto be blocked, as illustrated by the visual blockage 36. An alternativeapproach in the event of a visual blockage is to physically rotate thehead-mounted display 26 with respect to the user's line-of-sight, sothat the instantaneous viewport 22 is no longer centered around theuser's line-of-sight, but is instead skewed into an offset position 38.

[0065] In any of these embodiments, an improvement to the simplescrolling of the instantaneous viewport 22 can be achieved usinghardware or software logic that enables scrolling using a combination ofdata generated by the head-tracker 28 and the input controller 16.Specifically, when the mouse is not active, head-tracker 28 output isused to perform large-magnitude positioning of the instantaneousviewport 22 with respect to the virtual display 20. Once theinstantaneous viewport 22 is positioned at the approximate desiredposition using head movements, the input controller 16 can then be usedto perform fine positioning of the instantaneous viewport. The inputcontroller 16 can also be used to select data items, or click and dragto select multiple data items, as is common within the art. Wheneversuch actions are taken with the mouse, the instantaneous viewport ismoved appropriately, maintaining the mouse pointer 32 at its selectedlocation within the instantaneous viewport. In a preferred embodiment,the input controller 16 and head-tracker 28 can operate simultaneously,which allows “click & drag” functions such as holding down the mousebutton to anchor one corner of a selection box, then scrolling the headuntil an opposing corner is reached, and releasing the mouse button toselect all of the items within the resulting selection box.

[0066] The present invention has been implemented in two alternativeprototype embodiments, with additional embodiments contemplated. Thefirst embodiment is constructed using an Apple Macintosh Duo 230portable computer 12, a General Reality CyberEye Model 100 head-mounteddisplay 26, and InLARGE screen magnifier software by Berkeley Systems(Berkeley, Calif.). In this embodiment, the head-tracker 28 is anexperimental device utilizing Gyrostar ENC-05E solid-state gyroscopes byMurata Manufacturing Company (Kyoto, Japan, and US location at Smyrna,Ga.). Two gyroscopes are used, one each for the head's pitch (elevation)and yaw (direction) degrees of freedom. The output of each gyroscopeconsists of a differential voltage, with the difference voltage directlyproportional to the angular velocity of the sensor. These outputs arefed to the Macintosh computer 12 via the Apple Desktop Bus (ADB) Port,which is used on all Macintosh computers for accepting input fromkeyboards, mice, and other input control devices. Because the gyroscopesoutput differential data representing an angular velocity, the data isdigitized using a simple analog-to-digital converter integrated circuit,and then used directly for scrolling the imagery, with only a linearscaling factor applied. This scaling factor is dependent on themagnification factor applied to the imagery, and serves to maintain theenlarged image at a fixed position in space as perceived by the user. Inthe case of an absolute orientation tracker such as a magnetometer, thedata must first be converted from orientation to rate of change inorientation by taking the mathematical derivative of the data withrespect to time.

[0067] In this first embodiment and most conceivable alternativeembodiments which utilize differential head tracking devices such asgyroscopes and accelerometers, various tracking errors are introduced bythe lack of a stable reference. These errors include drift, temperatureinstability, hysteresis, cross-axis coupling, and limited dynamic range.

[0068] Drift is evidenced by slow motions in the imagery which occur inthe absence of any true head motion, and is corrected by incorporating alow-frequency cut-off filter in the tracking data output. Suchlow-frequency cut-off filters are well-known in the tracking art, and donot affect perceived performance.

[0069] Temperature instability is evidenced by drift that occursfollowing rapid changes in the ambient temperature in which the trackeris used. Some such instability is removed with software which acts likea low-frequency cut-off filter by ignoring D.C. drift, while some isunavoidable and requires a waiting period for temperature of the systemhardware to stabilize. This software ignores any D.C. signal componentfrom the head tracker 28 and allows a scaling factor to be input to thesystem to control the magnitude of the shift in the virtual image as afunction of the amount of rotation of the user's head.

[0070] Hysteresis is evidenced by sensitivity differences between motionin one direction and motion in a direction 180 degrees opposite. Thisartifact can be addressed by using a different scaling factor dependingupon the tracker's direction of travel. The magnitude of this sealingfactor can be determined experimentally, depending upon the magnitudeand direction of the hysteresis.

[0071] Cross-axis coupling is evidenced by the displayed image moving asmall amount in one axis when all of the head motion is along anorthogonal axis. This artifact is also controlled by the software whichacts like a low-frequency cut-off filter, and may be further controlledby disabling one axis whenever the orthogonal axis rate of motion isgreater than an empirically-determined threshold.

[0072] Finally, dynamic range limitations result in upper and lowerlimits to the rate at which the head may be turned while stillmaintaining the perception that the image is fixed in space. The lowerlimit is nominally determined by the electronic noise floor of thesensor devices, although it is raised by addition of the low-frequencycut-off filter. The upper limit is determined by the maximum rate ofchange measurable by the sensor. If this rate of change is exceeded byoverly rapid turning of the user's head, the imagery will appear to movein the same direction as the head is turning. This last artifact has notbeen solved, but may be addressed in a future embodiment through the useof an absolute position tracker.

[0073] In this first embodiment, the Apple Macintosh ADB port allowssimultaneous operation of multiple input control peripherals. Because ofthis feature, either the input controller 16 or a variety of secondarycontrollers may be used in conjunction with the head-tracker 28 toperform navigation within the imagery. Such controllers includejoysticks, trackballs, light pens, simple switches, or any other controldevice which is ADB port compatible.

[0074] The second embodiment of the invention has been implemented forthe Intel/Microsoft personal computer architecture. In this embodiment,the computer 12 is a 90 Mhz Pentium host computer, the head-mounteddisplay 26 is a CyberEye Model 100, and the head-tracker 28 is a 3-axismagnetometer, available as the Model TCM-2 from Precision Navigation,Inc. (Mountain View, Calif.) or the CyberTrack™ from General RealityCompany (San Jose, Calif.). This embodiment has been made functionalusing LP-DOS from Optelec (Westford, Mass.) as the screen enlarger 24,although alternative commercially available screen enlargers may be usedwithout modifying the remaining components of the system.

[0075] In this second embodiment, the selected head-tracker 28 is anabsolute orientation sensor, although any alternative head-trackingdevice may be used. The specific 3-axis magnetometer used as thehead-tracker 18 in this embodiment connects to the serial port of thecomputer 12, and provides an internal conversion from absolute positionto differential data in the form of mouse “mickeys” compatible with theIntel/Microsoft personal computer architecture. Because of this feature,the output of the head-tracker 28 can be read directly by a standardMicrosoft mouse driver, which provides a menu for setting the scalingfactor required for maintaining a fixed image as perceived by the user.

[0076] In this second embodiment, the Intel/Microsoft personal computerarchitecture does not make use of the Apple ADB bus, but instead usesRS-232 serial communication ports to connect to control devices such asthe head-tracker 28 and the input controller 16. This complicates thesystem design because the standard Microsoft mouse driver can onlyaccess one serial port (and therefore one control device) at anyparticular moment. Since proper operation of the invention 10 requiressimultaneous response to the head-tracker 28 and the input controller16, hardware or software is required to access two control devicessimultaneously.

[0077] In the most common case of a conventional computer mouse employedas the input controller 16, this may be accomplished in one of at leastfive ways. First, an existing mouse driver that includes dual-portcapability such as the original Borland mouse driver may be used.Second, the source code for the standard Microsoft mouse driver may bemodified to support simultaneous access to two serial ports. Third, adevice such as the “WhyMouse” by P.I. Engineering (Williamston, Mich.)may be used. This device serves as a “Y” adapter to connect twomouse-type pointing devices into one serial port. Circuitry internal tothe WhyMouse automatically routes one or the other device's data to theserial port based on a priority scheme, wherein the first device-to emitdata gains control of the input. Fourth, a custom solution can beimplemented in the form of a unique software driver, or fifth, in theform of a software module running on an intelligent input/outputcontroller such as the Rocketport32 card from Industrial Computer Source(San Diego, Calif.). Such intelligent input/output controllers areavailable from several commercial sources in the form of a circuit boardthat may be inserted in an expansion slot within the computer 12. Thesecircuit boards include two or more serial ports, as well as an on-boardprocessor that can manipulate the inputs from the serial ports prior todelivering the tracking data to the computer's internal bus.

[0078] Of the four potential approaches to dual input device operation,the preferred embodiment exploits the fourth approach. This is because acustom software module avoids hardware costs, while providing thegreatest flexibility in terms of application optimization and userconvenience. For example, a custom software module allows the user toselect whether the input controller 16 and head-tracker 28 can operatesimultaneously in the manner-preferred by the inventor, or whether theinput controller 16 and head-tracker 28 operate in a priority scheme asprovided in the WhyMouse product. In addition, a custom softwareapproach can provide optional use of a variety of alternative devices asthe input controller 16. For example, some users may prefer ahand-operated joystick to a mouse, while physically-impaired users mayrequire a finger-operated joystick or head-operated directional switcheswith a puff & suck switch for activating the mouse clicking function.

[0079] Referring to FIG. 5, a block diagram of an advanced embodiment ofthe invention 10 is shown. In FIG. 5, the computer 12, keyboard 14,input controller 16, screen enlarger 24, head-mounted display 26, andhead-tracker 28 are illustrated as previously defined, while a standardcomputer operating system such as Microsoft Windows is conceptuallyshown as 42, a typical computer application such as Microsoft Word isshown as 44, and a typical display driver such as a VGA graphics boardis shown as 46. The software module used for combining the inputs of theinput controller 16 and head-tracker 28 is shown as the control driver48. An additional software module called the input remapper 50 is alsoshown interposed between the input controller 16 and the control driver48. This input remapper 50 is a program that converts inputs from avariety of potential devices that may be used as the input controller 16into a single convenient data format such as mouse mickeys. For example,the output of a joystick used as the input controller 16 can be remappedby the input remapper 50 so that pressing the joystick trigger buttonresults in a mouse click signal being sent to the control driver 48,moving the joystick to the left results in emulation of sliding themouse to the left, etc. By separating the input control software into acontrol driver 48 and an input remapper 50, the control driver 48 can bemade standard, with only the input remapper 50 modified whenever it isdesirable to support a new type of input controller 16 within theinvention 10. The use of an input remapper 50 is a common approach inCD-ROM personal computer games, where the user can select between themouse, joystick, keyboard, or other devices for purposes of controllinggame play.

[0080]FIG. 5 also illustrates use of a tracking formatter 52, which is asoftware module interposed between the head-tracker 28 and the controldriver 48. The tracking formatter 52 performs various functionsdepending upon the particular sensing means employed within thehead-tracker 28. These functions can be separated into three categories.

[0081] The first category of functions performed by the trackingformatter 52 is conversion of the data stream emanating from thehead-tracker 28 into a format readable by the control driver 48. Thisconversion is tracking sensor-dependent. In the case of amagnetometer-based tracker with mouse emulation as used in theIntel/Microsoft embodiment, no conversion is required. In the case of amagnetometer without mouse emulation, the tracking output would consistof rapidly-updated azimuth and elevation position figures, in whichevent the tracking formatter 52 would subtract the prior position samplefrom the present sample and then convert the format to mouse mickeys toprovide the control driver 48 with emulated mouse output consisting ofchanges in position. In the case of a gyroscopic tracker—with outputconverted to digital form such as that used in the Apple Macintoshembodiment, the output of the head-tracker 28 consists of angularvelocity figures. In this event, the angular velocity samples are simplymultiplied by the time period of each sample to yield a change inposition, with each positional change then converted into mouse mickeysby the tracking formatter 52.

[0082] The second category of functions performed by the trackingformatter 52 consists of error correction functions such as thosepreviously described for the Apple Macintosh embodiment of the invention10. In that embodiment, the tracking formatter 52 performs low-frequencycut-off filtering, applies a directionally-dependent scaling factor, anddisables one axis of travel when the orthogonal axis velocity risesabove a threshold. These functions could also be performed in hardwaresuch as an application-specific integrated circuit or afield-programmable gate array if higher-performance at high-volumeproduction is desirable.

[0083] The third category of functions performed by the trackingformatter 52 consists of enhancement functions such as orientationprediction. This function addresses the pipeline delay between theinstant in time when the head is turned, and the time when the displayedimage is updated to display the new user line-of-sight. This delay canbe calculated to be the sum of the tracker sensing time, tracker tocomputer communication time, tracker formatter processing time, controldriver processing time, operating system and application softwareprocessing time, screen enlarger processing time, and display refreshtime. In a typical embodiment, the sum of these delays can becomebothersome, causing a perception of the display “swimming” with respectto the user's line of sight changes. This swimming causes perceptualmismatches between the user's internal proprioceptive cues and externalvisual cues, which in severe cases can cause disorientation and nauseaeffects known in the virtual reality field as simulator sickness. Toavoid such effects, the current position and velocity of the head ineach degree of freedom can be used to predict the future position, inthe manner of So and Griffin or Azuma and Bishop. By doing so, thepredicted future position can be used as the input to the processingpipeline instead of the current actual position, thus decreasing theaverage mismatch between the proprioceptive and visual cues.

[0084]FIG. 5 also illustrates the use of a voice recognition system as ameans for inputting control commands and application data into theinvention 10. The voice recognition system consists of a microphone 54disposed near the user's mouth, such as by mounting onto or within thehead-mounted display. The output of the microphone is input to thecomputer's audio input port, which digitizes the audio data. The digitaldata is then analyzed by a voice recognizer 56, which may consist ofhardware, software, or a combination of the two. For example, a typicalembodiment-of the voice recognizer 56 for an Intel/Microsoftarchitecture would consist of Dragon Dictate software by Dragon Systems(Newton, Mass.), running on a SoundBlaster audio board by CreativeLaboratories (Milpitas, Calif.). Regardless of the particular embodimentof the voice recognizer 56, the output is sent to the operating systemin the form of digital data interpreted as either commands or contentdepending upon the state of the operating system.

[0085] The incorporation of the voice recognizer 56 enables use ofconvenience-enhancing commands for purposes such as positioning thevirtual display with respect to the user's line-of-sight, selectingenlargement factors, controlling tracking, selecting between systemoperating modes, and controlling individual computer applications. Forexample, position commands include “center me” to center the user'sinstantaneous viewport 22 within the virtual display 20, “top right” tomove the instantaneous viewport 22 to the top right, etc. Enlargementcommands include absolute commands such as “Mag 8” to set the screenenlarger 24 to a magnification of 8 to 1, and relative commands such as“zoom double” to temporarily increase the magnification by a factor oftwo. Tracking control commands include “lock vertical” to lock-outresponse to the elevation tracking function, which simplifies scrollinghorizontally across text. Selecting between system operating modesincludes a complete set of commands for operating the screen enlarger24, such as “scroll text” to, enter the enlarger's text scrolling mode.Finally, application control commands are application-dependent andavailable commercially as libraries, which typically include most or allmouse-accessible functions such as “Page down”, “font: Times”, “edit:cut”, etc.

[0086]FIG. 5 additionally illustrates a spatialized audio generator 58,which is used to alert the user to computer-generated events occurringoutside the user's instantaneous viewport 22. This is done by providingthe user with slightly different signals in each ear via a pair ofloudspeakers or stereo headphones 60, with the differences calculated tosimulate directionality via slight delays between the nearer ear'ssignal and the farther ear's signal, slight reduction in high-frequencycontent in the farther ear's signal, and other spatial processing as iscommonly known in the art. The spatialized audio generator 58 can beconstructed from commercially-available components such as aSoundBlaster audio board from Creative Laboratories (Milpitas, Calif.),which includes audio spatialization software as a standard feature. Theinput to the spatialized audio generator 58 is provided by the operatingsystem 42 for simple alerts such as “beeps” signifying an error or othermessage window, and may be provided by the application software 44 orthe screen enlarger 24 for more advanced messages such as synthesizedvoice messages or text-to-speech conversion.

[0087] In FIG. 5, it is noted that the control driver 48 contains ascaling factor used to adjust the amount by which the instantaneousviewport 22 moves across the virtual display 20 per degree of headrotation. In most instances, this scaling factor is set so that thevirtual display 20 appears fixed in space while the instantaneousviewport is scanned across it. However, for extremely high magnificationfactors, fixing the virtual display can be problematic, as the user'shead may be required to rotate more than is comfortable to scan from oneedge of the virtual display to the opposing edge. Under such conditions,the present invention 10 may be configured by the user with a differentscaling factor, which increases the amount by which the instantaneousviewport 22 moves across the virtual display 20 for each degree of headrotation. When viewed by the user, this results in the virtual displayappearing to move across the user's instantaneous viewport 22 in adirection directly opposite to that in which the user is scanning.*Because the instantaneous viewport 22 and the virtual display are bothmoving in opposite directions, scrolling appears to be faster, but theuser can scan from one edge of the virtual display 20 to the opposingedge with a smaller total head rotation. This approach to utilizing thepresent invention is deemed spatial field compression.

[0088] It is also noted in FIG. 5 that a snap-back function may beincluded within the control driver 48, wherein data from the inputcontroller 16 is used only for temporary repositioning of the mousepointer 32 and instantaneous viewport 22 within the virtual display 20.Specifically, this function records activity of the input controller 16while such activity is being used to control the displayed imagery. Oncesuch activity ceases, the inverse of the recorded activity is fed to theoperating system 42, which snaps-back the image displayed in theinstantaneous viewport 22 to that which would be viewed in the absenceof the input controller 16. The result of this snap-back function isthat the virtual display 20 is maintained at a fixed location in space,which may be temporarily modified by use of the input controller 16, butis returned to following use of the input controller 16.

[0089] It is also noted in FIG. 5 that additional image processing maybe performed by the screen enlarger 24 or elsewhere in the processingpipeline to incorporate additional functions which may be desirable forvisually-impaired users or other applications. For example, a commonfeature in commercial screen enlargers consists of contrast reversal,where instead of displaying black text on a white background, white textcan be displayed on a black background. This improves text readabilityfor some users. Another potentially useful feature is image enhancement,wherein the imagery is digitally enhanced to strengthen edges, whichimproves resolution ability for some users.

[0090] Finally, in FIG. 5 it is noted that if the screen enlarger 24 isset to an enlargement factor of one-to-one or omitted entirely, and adisplay driver 46 providing a virtual desktop function such as the MGAMillenium by Matrox (Dorval, QC, Canada) is used., then the presentinvention 10 can be used in an identical fashion by anon-visually-impaired user for purposes of accessing large areas of avirtual desktop, which enhances tasks such as simultaneous use of manyindividual computer applications.

[0091] A further embodiment of the present invention 10 is illustratedin FIG. 6, which shows the invention 10 applied to magnification ofreal-time imagery. In this embodiment, a video camera 62 and a videoframe grabber board 64 are added to any of the previously describedembodiments. The video camera is then mounted in a stationary positionand aimed at an image to be enlarged 66.

[0092] This image to be enlarged 66 may be a small object to bemagnified such as text in a book, or may be a distant object to beresolved such as a blackboard in a classroom lecture. Each frame ofvideo captured by the video grabber board 64 is output to the system busas application output data by software included commercially with thevideo frame grabber board 64, and fed to the screen enlarger 24. Thescreen enlarger 24 magnifies the imagery, creating a virtual display 20of the image to be enlarged 66 that occupies a larger angular extent asseen by the user than does the image to be enlarged 66. The head-mounteddisplay 26, head-tracker 28, tracking formatter 52, and control driver48 are then used as previously described to provide an instantaneousviewport 22, which may be positioned at any convenient point within thevirtual display 20 by turning one's head. In this embodiment,improvement is made upon earlier closed-circuit television inventionsfor the visually-impaired in that the camera captures the entire imageto be enlarged 66 at all times, instead of moving with the user's heador hand and capturing just the amount of imagery that can be displayedwithin the instantaneous viewport 22. By doing this, spatial awarenessis maintained, but jitter in camera motion is not magnified to becomedisruptive to the user. In addition, by interposing a computer withinsuch a closed-circuit television loop, any image may be instantly savedto permanent memory with a single keystroke for later review, editing,or printout.

[0093] While the invention has been shown and described with referenceto a particular set of embodiments, it will be understood by thoseskilled in the art that various alterations and modifications in formand detail may be made therein. Accordingly, it is intended that thefollowing claims cover all such alterations and modifications that mayfall within the true scope and spirit of the invention.

What is claimed is:
 1. A device for interacting with a computer, saiddevice comprising: an input device for receiving user input, said inputdevice including at least one accelerometer for receiving said input inthe form of acceleration of at least a portion of said input device; anda signal processing device for producing at least one computer commandin response to said input.
 2. The device of claim 1, wherein saidacceleration input includes torsional acceleration about at least oneaxis of rotation.
 3. The device of claim 1, wherein said accelerationinput includes substantially linear acceleration in at least onedirection.
 4. The device of claim 2, wherein said input device operatesindependently of gravitational forces.
 5. The device of claim 1, whereinsaid input device is incorporated in a handheld computer.
 6. The deviceof claim 1, wherein said computer command is a game command for acomputer game.
 7. A controller for controlling an electronic devicecomprising: at least one sensor for sensing user input, wherein saidinput includes acceleration in at least one direction; at least oneprocessor for generating a signal in response to said input; and acommunications link for providing said signal to said electronic devicein order to control said electronic device.
 8. The controller of claim7, wherein said electronic device includes a computer game station. 9.The controller of claim 7, wherein said acceleration in at least onedirection includes torsional acceleration about at least one axis ofrotation.
 10. The controller of claim 7, wherein said electronic deviceincludes a visual display showing a first page, and said electronicdevice displays a second page in response to said user input.
 11. Thecontroller of claim 7, wherein said electronic device includes a displayfor viewing at least a portion of a full view, wherein said electronicdevice is operative for viewing other portions of said full view inresponse to said user input.
 12. A controller for an electronic device,said controller including a set of one or more motion sensors, each ofwhich respond electrically to motion of at least a portion of saidelectronic device; an information collection device for collectinginformation from said set of one or more motion sensors, wherein saidinformation collection device collects electrical information, said setof one or more motion sensors capable of detecting motion in at leastone dimension; a command device which interprets said electricalinformation collected on said information collection device, saidcommand device including command logic and a store of control commands,wherein said information collected is interpreted by said command logicby comparing said information with one of said store of controlcommands, said control commands being in a format which corresponds toat least one motion; and wherein said command logic determines which oneof said set of control commands will be executed on said device.
 13. Thecontroller of claim 12, wherein said at least one motion includes a setof motions corresponding to at least one control command.
 14. Thecontroller as recited in claim 12, where said command logic determineswhich portion of a virtual desktop will be presented on a physicaldisplay device.
 15. The controller of claim 12, wherein said displaydevice and said electronic device are integrated as part of a handheldportable computer.
 16. The controller as recited in claim 12, whereinsaid controller further comprises motion filtering logic, wherein saidlogic removes motion below a set threshold from said motion calculationlogic.
 17. The controller as recited in claim 16, wherein said thresholdis filtered by said set of one or more motion detectors.
 18. Thecontroller as recited in claim 16, wherein said threshold is filtered bysaid information collection device.
 19. The controller as recited inclaim 16, wherein said threshold is filtered by said command device. 20.The controller as recited in claim 12, wherein one of said set of motionsensors respond to motion towards and away from a user of said portabledevice.
 21. The controller as recited in claim 12, wherein one of saidset of motion sensors respond to motion in a vertical plane relative toa user of said portable device.
 22. The controller as recited in claim12, wherein one of said set of motion sensors respond to motion in asubstantially horizontal plane relative to a user of said portabledevice.
 23. The controller as recited in claim 12, wherein one of saidset of motion sensors respond to an angle corresponding to a verticalplane perpendicular to a sightline and in front of a user and an angleof said device.
 24. The controller as recited in claim 12, wherein oneof said set of motion sensors respond to angular motion about asightline substantially perpendicular to a vertical plane substantiallyfacing a front of a user.
 25. The controller as recited in claim 12,wherein one of said set of motion sensors is an accelerometer.
 26. Thecontroller as recited in claim 12, wherein one of said set of motionsensors is a gyroscope.
 27. The controller as recited in claim 12,wherein said set of motion sensors include both an accelerometer and agyroscope.
 28. The controller as recited in claim 12, wherein saidcontrol command to be executed may be executed by an operating system.29. The controller as recited in claim 12, wherein said control commandto be executed may be executed by a software program running on saiddevice.
 30. The controller as recited in claim 12, wherein said controlcommand to be executed may be executed by a command located in an ASIC.31. The controller as recited in claim 12, wherein said control commandto be executed may be executed by a video driver running on said device.32. The controller as recited in claim 12, wherein said control commandto be executed may be executed by an adapter, said adapter fortranslating said control command to a command which may be executed bythe operating system of said device.
 33. The controller as recited inclaim 12, wherein said device is immunersive.
 34. The controller asrecited in claim 12, wherein said device includes display goggles, whichdisplay a virtual desktop.
 35. The controller as recited in claim 12,wherein said device is worn on a head.
 36. A method for controlling aportable electronic device by moving said device, comprising the stepsof sensing at least one acceleration of said device in at least onedirection; and providing at least one command to said device in responseto said at least one acceleration.
 37. The method of claim 36, furthercomprising: producing a signal in response to said at least oneacceleration; comparing said signal to a data base of stored signals andcorresponding commands in order to determine an appropriate commandcorresponding to an intended user input.
 38. The method of claim 36,further comprising: filtering out said signals that do not correspond toone of said stored commands, such that no unintended command is executedon said device.
 39. A method for controlling a computer game program,comprising: sensing at least one acceleration of a control device in atleast one direction; and providing at least one game command to saidcomputer game program in response to said at least one acceleration. 40.The controller of claim 7, wherein said electronic device includes adisplay for viewing at least a portion of a full view, wherein saidelectronic device is operative for viewing other portions of said fullview in response to said user input.
 41. The controller of claim 1,wherein said accelerometer is operative to sense an accelerationmagnitude.
 42. The cotroller of claim 7, wherein said electronic deviceincludes a personal digital assistant (PDA).